The present invention relates to a method of describing image information. In particular, the present invention relates to a method of describing thumbnail information on thumbnail frames which are obtained by sampling video frames with arbitrary time intervals and in spatially arbitrary size, and video retrieval and video reproducing methods and apparatus employing the thumbnail information.
In recent years, with advancement of semiconductor technology and digital signal processing technology, it has been possible to perform processing for converting moving image (video) information from analog data to digital data and compressing the digital data in real time. Actually, in digital satellite broadcasting, digital video data compressed and encoded by MPEG-2 that is the international standard for moving picture image compression is distributed, and the compressed video data is decompressed and decoded at each home in real time so that cinemas or the like can be watched by means of a television receiver.
In addition, with achievement of a high-density optical disk, a technology for recording digital video data compressed by MPEG-2 or the like is becoming at a practical stage. A typical example of such optical disk medium includes DVD-RAM or CD-RW. Although a recording time is shorter than that of DVD-RAM, it is also possible to record digital video data in HDD. Further, it is considered that digital video data recorded in the DVD-RAM or the like is required so as to easily retrieve as in digitized texts or still picture data.
A classical technique for video retrieval is that a title name and a keyword are defined for each video file such as cinema, and are retrieved based on one or both of the title name and keyword. This method is disadvantage in that retrieval itself is easy, but detailed retrieval according to the content of a video cannot be done, and whether or not a predetermined video is obtained cannot be identified as long as the video is actually reproduced and displayed.
In recording the compressed digital video data, the moving video image can be handled as continuity of still image frames. Thus, a method for selecting and listing a characteristic image frame called a typical frame from an original video by means of an image processing technology is considered. As a typical frame, a frame called scene change at which a scene is switched may be often employed. However, such scene change does not occur once per several seconds, occasionally once per some tens of seconds, and thus, there is a limitation to expressing the content of the video at the typical frame. If an attempt is made to check contents of frames between scene changes, the original video data must be decoded and displayed.
The digital video data compressed in accordance with the international standards such as MPEG-1 and MPEG-2 includes a mechanism for random access to a certain extent so that variable speed reproduction (trick play) such as fast reproduction can be performed. However, these variable speed reproductions are heavy in processing because they are performed by manipulating digital video data itself, and a burden upon processing is increased in a receiving device for home use with its small computer power. In addition, as in browser in video on demand or Internet, when variable speed reproduction is performed in environment such that digital video data is distributed from a server installed at a remote site through a network, and the distributed data is received by a computer or a television receiver at home, there is difficulty that a network traffic is increased.
As described above, a conventionally general video retrieval is such that information is retrieved based on a title name or a keyword assigned to a video file, and in reality, environment in which the content of a video is checked and retrieved is not sufficiently provided.
In addition, there is a problem that a method for selecting a portion of scene change from an original video as a typical frame and listing the typical frame is incapable of checking contents of video frames between scene changes.
Further, in a mechanism for variable speed reproduction incorporated in the international standard for moving image compression such as MPEG-1 or MPEG-2, variable speed reproduction is performed by manipulating digital video data itself. Thus, a burden upon processing is increased in a small-sized device with its computer power. In addition, when an attempt is made to perform variable speed reproduction in the environment such that digital video data distributed through a network is received, there has been a problem that a network traffic is increased.